Contrastive Learning Reduces Hallucination in Conversations

Pre-trained language models (LMs) store knowledge in their parameters and can generate informative responses when used conversational systems. However, LMs suffer from the problem of “hallucination:” they may plausible-looking statements that are irrelevant or factually incorrect. To address this problem, we propose a contrastive learning scheme, named MixCL. A novel mixed objective is proposed to explicitly optimize implicit elicitation process LMs, thus reduce hallucination conversations. We also examine negative sampling strategies retrieved hard negatives model-generated negatives. conduct experiments on Wizard-of-Wikipedia, public, open-domain knowledge-grounded dialogue benchmark, assess effectiveness MixCL effectively reduces conversations achieves highest performance among LM-based agents terms relevancy factuality. show comparable state-of-the-art KB-based approaches while enjoying notable advantages efficiency scalability.